The present invention is directed to an augmented imaging system with tracking capability. In an augmented imaging system virtual images are rendered over a real-world background. In an augmented retinal imaging system, for example, the real world background light is viewed while virtual imagery is scanned onto a viewer's retina. The viewer sees the virtual imagery among the real world background scene.